Skip to main content

Overview

The EDL Pipeline integrates with multiple Dhan ScanX API endpoints to fetch comprehensive market data. All endpoints use POST method with JSON payloads unless specified otherwise.

Authentication & Headers

All Dhan API requests use standardized headers from pipeline_utils.py:
headers = {
    "Content-Type": "application/json",
    "User-Agent": "<Random User-Agent>",
    "Accept": "application/json, text/plain, */*",
    # Optional for some endpoints:
    "Origin": "https://scanx.dhan.co",
    "Referer": "https://scanx.dhan.co/"
}
The pipeline rotates between 5 different User-Agent strings to avoid detection:
  • Chrome on Windows 10
  • Chrome on macOS
  • Chrome on Linux
  • Firefox on Windows
  • Safari on macOS
Headers with Origin/Referer are required for OHLCV endpoints but optional for market data endpoints.

Market Data Endpoints

Full Market Data

URL
string
https://ow-scanx-analytics.dhan.co/customscan/fetchdt
Method
string
POST
Script
string
fetch_dhan_data.py
Output
string
dhan_data_response.json + master_isin_map.json
Retrieves all ~2,775 NSE stocks in a single call with comprehensive market data. Request Payload:
{
  "data": {
    "type": "full",
    "whichpage": "nse_total_market",
    "filters": [],
    "sort": "Mcap",
    "sorder": "desc",
    "count": 5000,
    "page": 1
  }
}
data
array
Array of stock objects containing Symbol, Name, ISIN, Sid, Market Cap, Price, Volume, and other market metrics
cURL Example:
curl -X POST https://ow-scanx-analytics.dhan.co/customscan/fetchdt \
  -H "Content-Type: application/json" \
  -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" \
  -d '{
    "data": {
      "type": "full",
      "whichpage": "nse_total_market",
      "filters": [],
      "sort": "Mcap",
      "sorder": "desc",
      "count": 5000,
      "page": 1
    }
  }'

Fundamental Data (Results & Ratios)

URL
string
https://open-web-scanx.dhan.co/scanx/fundamental
Method
string
POST
Script
string
fetch_fundamental_data.py
Timeout
string
30 seconds
Output
string
fundamental_data.json (35 MB)
Fetches quarterly results, ratios, and financial metrics for stocks. Supports batch requests with up to 100 ISINs per call. Request Payload (Single ISIN):
{
  "data": {
    "isin": "INE002A01018"
  }
}
Request Payload (Batch - up to 100 ISINs):
{
  "data": {
    "isins": [
      "INE002A01018",
      "INE040A01034",
      "INE467B01029"
    ]
  }
}
status
string
“success” or “error”
data
array
Array containing quarterly results, EPS, sales, OPM, debt ratios, and ownership data
Response Structure:
{
  "status": "success",
  "data": [
    {
      "isin": "INE002A01018",
      "Symbol": "RELIANCE",
      "Name": "Reliance Industries Ltd",
      "quarterly_results": [...],
      "ratios": {...},
      "ownership": {...}
    }
  ]
}
The pipeline batches requests in chunks of 100 ISINs with a 0.5-second delay between batches to be polite to the server.

Company Filings (Hybrid)

URL 1
string
https://ow-static-scanx.dhan.co/staticscanx/company_filings
URL 2
string
https://ow-static-scanx.dhan.co/staticscanx/lodr
Method
string
POST
Script
string
fetch_company_filings.py
Threads
number
20
Timeout
string
10 seconds per endpoint
Output
string
company_filings/{SYMBOL}_filings.json
Fetches regulatory filings from TWO endpoints and merges results for maximum coverage. Deduplicates by news_id + news_date + caption. Request Payload:
{
  "data": {
    "isin": "INE002A01018",
    "pg_no": 1,
    "count": 100
  }
}
data
array
Array of filing objects with news_id, news_date, caption, descriptor, and file_url (PDF link)
Deduplication Logic:
# Creates unique key from:
key = news_id if news_id else f"{news_date}_{caption}"

Live Announcements

URL
string
https://ow-static-scanx.dhan.co/staticscanx/announcements
Method
string
POST
Script
string
fetch_new_announcements.py
Threads
number
40
Timeout
string
10 seconds
Output
string
all_company_announcements.json
Request Payload:
{
  "data": {
    "isin": "INE002A01018"
  }
}
data
array
Array of announcement objects with events, date, and type fields
Response Example:
{
  "data": [
    {
      "events": "Board Meeting Intimation",
      "date": "2024-03-15",
      "type": "Company Update"
    }
  ]
}

Advanced Indicators (Pivot, EMA, SMA)

URL
string
https://ow-static-scanx.dhan.co/staticscanx/indicator
Method
string
POST
Script
string
fetch_advanced_indicators.py
Threads
number
50
Timeout
string
10 seconds
Output
string
advanced_indicator_data.json (8.3 MB)
Requires Sid (Security ID) from master_isin_map.json. Request Payload:
{
  "exchange": "NSE",
  "segment": "E",
  "security_id": "13",
  "isin": "INE002A01018",
  "symbol": "RELIANCE",
  "minute": "D"
}
exchange
string
required
“NSE” for National Stock Exchange
segment
string
required
“E” for Equity segment
security_id
string
required
Numeric Security ID (Sid) from master ISIN map
minute
string
required
“D” for Daily timeframe
data
array
Contains EMA, SMA, Technical Indicators, and Pivot Point arrays
Response Structure:
{
  "data": [
    {
      "EMA": [{"period": 20, "value": 2450.50, "status": "Above"}],
      "SMA": [{"period": 50, "value": 2380.20, "status": "Above"}],
      "Indicator": [{"name": "RSI", "value": 62.5, "sentiment": "Neutral"}],
      "Pivot": [{"type": "Standard", "pivot": 2420.00, "r1": 2450.00, "s1": 2390.00}]
    }
  ]
}

Market News Feed

URL
string
https://news-live.dhan.co/v2/news/getLiveNews
Method
string
POST
Script
string
fetch_market_news.py
Threads
number
15
Timeout
string
10 seconds
Page Size
string
50 items per stock (max tested: 100)
Output
string
market_news/{SYMBOL}_news.json
Request Payload:
{
  "categories": ["ALL"],
  "page_no": 0,
  "limit": 50,
  "first_news_timeStamp": 0,
  "last_news_timeStamp": 0,
  "news_feed_type": "live",
  "stock_list": ["INE002A01018"],
  "entity_id": ""
}
page_no
number
Pagination support (0-indexed)
limit
number
Number of news items (tested up to 100)
data.latest_news
array
Array of news objects with title, sentiment, publish date, and source
Response Structure:
{
  "data": {
    "latest_news": [
      {
        "news_object": {
          "title": "Reliance Q4 results beat estimates",
          "text": "Full news summary...",
          "overall_sentiment": "positive"
        },
        "publish_date": 1710518400,
        "category": "Earnings"
      }
    ]
  }
}
The endpoint returns 429 (rate limit) errors if overwhelmed. The script implements a 2-second backoff on rate limits.

Corporate Actions

URL
string
https://ow-scanx-analytics.dhan.co/customscan/fetchdt
Method
string
POST
Script
string
fetch_corporate_actions.py
Modes
string
History (2 years back) + Upcoming (2 months ahead)
Output
string
history_corporate_actions.json, upcoming_corporate_actions.json
Fetches dividends, bonus issues, stock splits, and rights issues. Request Payload (Historical):
{
  "data": {
    "type": "full",
    "whichpage": "corporate_action",
    "filters": [
      {
        "field": "CorpAct.ActDate",
        "op": "GT",
        "val": "2024-01-01"
      }
    ],
    "count": 5000,
    "page": 1
  }
}
Request Payload (Upcoming):
{
  "data": {
    "type": "full",
    "whichpage": "corporate_action",
    "filters": [
      {
        "field": "CorpAct.ActDate",
        "op": "LT",
        "val": "2024-05-01"
      }
    ],
    "count": 5000,
    "page": 1
  }
}

Circuit Stocks

URL
string
https://ow-scanx-analytics.dhan.co/customscan/fetchdt
Script
string
fetch_circuit_stocks.py
Page Size
number
500
Output
string
upper_circuit_stocks.json, lower_circuit_stocks.json
Fetches stocks hitting upper or lower circuit limits.

Bulk/Block Deals

URL
string
https://ow-static-scanx.dhan.co/staticscanx/deal
Method
string
POST
Script
string
fetch_bulk_block_deals.py
Page Size
number
50 (auto-paginates all pages)
Output
string
bulk_block_deals.json
Request Payload:
{
  "data": {
    "defaultpage": "N",
    "pageno": 1,
    "pagecount": 50
  }
}

Historical OHLCV

URL
string
https://openweb-ticks.dhan.co/getDataH
Method
string
POST
Script
string
fetch_all_ohlcv.py
Threads
number
15
Timeout
string
15 seconds
Start Date
string
215634600 (Oct 31, 1976 - forces maximum history)
Interval
string
“D” (Daily candles)
Output
string
ohlcv_data/{SYMBOL}.csv
Request Payload:
{
  "EXCH": "NSE",
  "SYM": "RELIANCE",
  "SEG": "E",
  "INST": "EQUITY",
  "SEC_ID": "13",
  "EXPCODE": 0,
  "INTERVAL": "D",
  "START": 215634600,
  "END": 1710518400
}
INTERVAL
string
“D” for Daily, “W” for Weekly, “M” for Monthly
START
number
Unix timestamp for start date (215634600 = Oct 31, 1976)
END
number
Unix timestamp for end date (current time)
data.Time
array
Array of timestamps or date strings
data.o
array
Open prices
data.h
array
High prices
data.l
array
Low prices
data.c
array
Close prices
data.v
array
Volume
Response Example:
{
  "data": {
    "Time": ["2024-03-01", "2024-03-04", "2024-03-05"],
    "o": [2450.00, 2465.50, 2472.30],
    "h": [2480.00, 2490.20, 2485.60],
    "l": [2445.00, 2460.00, 2468.00],
    "c": [2475.50, 2478.80, 2481.20],
    "v": [15000000, 18500000, 16200000]
  }
}
This endpoint fetches data in 180-day chunks to avoid API limits. The script merges chunks and deduplicates by date.

Standalone Endpoints (Not in Pipeline)

FNO Stocks

Script
string
fetch_fno_data.py
Filter
string
FnoFlag=1, count: 500
Output
string
fno_stocks_response.json
Fetches 207 F&O eligible stocks.

FNO Lot Sizes

Script
string
fetch_fno_lot_sizes.py
Source
string
dhan.co/nse-fno-lot-size/ (Next.js data)
Output
string
fno_lot_sizes_cleaned.json

FNO Expiry Calendar

Script
string
fetch_fno_expiry.py
Source
string
dhan.co/_next/data/{buildId}/fno-expiry-calendar.json
Output
string
fno_expiry_calendar.json

All Indices

Script
string
fetch_all_indices.py
Source
string
customscan/fetchdt + Google Sheets Gviz fallback
Output
string
all_indices_list.json
Fetches 194 market indices.

ETF Data

Script
string
fetch_etf_data.py
Filter
string
ETFFlag, count: 1000
Output
string
etf_data_response.json
Fetches 361 ETFs.

Best Practices

Always include proper headers with User-Agent rotation to avoid detection.
Batch requests when possible (fundamental data supports 100 ISINs per call).
Add delays between batches (0.5s recommended) to be polite to servers.
Handle rate limits gracefully with exponential backoff on 429 errors.
Validate responses by checking status codes and response structure before parsing.